Face liveness detection is an important part of the face recognition process, and is particularly important for the security of identity verification. In view of the cheating methods such as photo, video, mask, hood and head model in the face recognition process, the RGB map and depth map information of the face was collected by the Intel Realsense camera, and a lightweight liveness detection of feature fusion was proposed based on MobileNetV3 to fuse the features of the depth map and the RGB map together and perform the end-to-end training. To solve the problem of large parameter quantity in deep learning and the distinction of the weight areas by the network tail, the method of using Streaming Module at the network tail was proposed to reduce the quantity of network parameters and distinguish weight regions. Simulation experiments were performed on CASIA-SURF dataset and the constructed CQNU-LN dataset. The results show that, on both datasets, the proposed method achieves an accuracy of 95% with TPR@FPR=10E-4, which is increased by 0.1% and 0.05% respectively compared to ShuffleNet with the highest accuracy in the comparison methods. The accuracy of the proposed method reaches an accuracy of 95.2% with TPR@FPR=10E-4 on the constructed CQNU-3Dmask dataset, which is improved by 0.9% and 6.5% respectively compared to those of the method training RGB maps only and the method training depth maps only. In addition, the proposed model has the parameter quantity of only 1.8 MB and FLoating-point Operations Per second (FLOPs) of only 1.5×10 ⁶. The proposed method can perform accurate and real-time liveness detection on the extracted face target in practical applications.